Pandas Utils

The Python dxapi client offers a utility module for integration with pandas. This module enables efficient reading of data from TimeBase into a pandas DataFrame and storing it from the DataFrame into stream.

Reading data to DataFrame

Utilize the pandas_utils.read_frame function from the dxapi to import data from streams or QQL query into a pandas.DataFrame.

import dxapi
from dxapi import pandas_utils

try:
    db = dxapi.TickDb.createFromUrl("dxtick://localhost:8011")
    db.open(True)
    
    fields=['open', 'close', 'high', 'low'] # Specify fields to read from streams
    symbols=['IBM', 'AAPL'] # Use None for all symbols
    types=None # Use None for all types
    df = pandas_utils.read_frame(db,
                                streams=['Daily'],
                                fields=fields,
                                tickers=symbols,
                                types=types,
                                from_time_str='2001-01-01',
                                to_time_str='2001-05-02')
    print(df)
finally:
    if (db.isOpen()):
        db.close()

Valid arguments:

• db (TickDb) TimeBase client instance;
• streams (list[str]) List of streams to read data from;
• query (str) QQL query to select data;
• types (list[str]) type message filter (None means select all types);
• tickers (list[str]) symbols filter (None means select all symbols);
• fields (list[str]) fields list to select;
• from_time (datetime) start time filter;
• to_time (datetime) end time filter;
• from_time_str (str) same as 'from_time', but in string format (ex, 2023-01-23 or 2023-01-23 10:30:00 UTC);
• to_time_str (str) same as 'to_time', but in string format (ex, 2023-01-23 or 2023-01-23 10:30:00 UTC).

If the query parameter is specified, data will be read from the QQL result set; otherwise, it will be retrieved from the streams list.

Write DataFrame to stream

Employ pandas_utils.write_frame in conjunction with pandas_utils.bind_frame. The pandas_utils.bind_frame function enables you to construct DataFrame to stream bindings, thereby controlling how columns should be matched to the stream's fields.

import dxapi
from dxapi import pandas_utils

try:
    db = dxapi.TickDb.createFromUrl("dxtick://localhost:8011")
    db.open(False)
    
    fields=['open', 'close', 'high', 'low']
    symbols=['IBM', 'AAPL'] # Use None for all symbols
    types=None # Use None for all types
    df = pandas_utils.read_frame(db,
                                    streams=['Daily'],
                                    fields=fields,
                                    tickers=symbols,
                                    types=types,
                                    from_time_str='2001-01-01',
                                    to_time_str='2001-05-02')
    print(df)
    
    # Write DataFrame to new stream
    binding = pandas_utils.bind_frame(db, 'Daily_new', df)
    
    # change bindings if need
    pandas_utils.write_frame(db, binding, df)
finally:
    if (db.isOpen()):
        db.close()

Download examples.